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ABSTRACT 

The work presented in this paper defin.ed conceptually 
the concepts of multidimensional discrimination and information, 
derived mathematical expressions for the concepts for a particular 
multidimensional item response theory (IRT) model, and applied the 
concepts to actual test data. Multidimensional discrimination was 
defined as a function of the slope of the item response surface in 
the direction specified by the multidimensional difficulty. For the 
multidimensional extension of the two-parameter logistic (MLPL) 
model, this definition resulted in a statistic that had the same 
relationship to multidimensional difficulty that "a" did to "b" for 
the unidimensional IRT models. This statistic was defined as the 
square root of the sum of squared a-parameters from the MLPL. 
Multidimensional information was defined by the same mathematical 
function as unidimensional information, but the directional 
derivative was substituted for the standard derivative in the 
numerator of the information expression. The use of the directional 
derivative resulted in rn information measure that corresponded to a 
direction in ability space. The American College Testing Program's 
ACT Assessment Mathematics Usage Test was used to demonstrate the 
multidimensional discrimination and information. The multidimensional 
measures of item quality, item precision, and test precision provide 
tools to gain a better understanding of the measurement process. 
(PN) 
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The Discriminating Power of Items 
that Measure More than One Dimension 

Statistical measures of item discrimination are computed for 
several reasons. The traditional measures, usually the point- 
biserial or biserial correlations between item scores and total 
scores, are used as general indicators of item quality, or as 
screening variables for use in selecting items for a test. During 
test construction, items are often selected to have a 
discrimination index that is greater than a specified value, such 
as .20 or .30. Discrimination indices may also be used to 
determine whether an item is measuring the same construct as the 
total score on the test. For the most part, the newer, item 
response theory (IRT) measures of discrimination and the related 
concept, information, are used in the same way, but they are also 
used to specify the precision of measurement provided by an item 
at different levels of ability along the ability scale. 

Both the traditional and IRT measures of the discriminating 
power of an item are based on the assumption that a test measures 
a single trait--either that defined by the total score, or by 
the e-scale. This paper will generalize the concept of item 
discrimination to the case where more than one ability is required 
to determine the correct response to an item. In making this 
generalization, the conceptual framework supplied by IRT will be 
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used. This paper will also draw upon the definition of 
multidimensional item difficulty (MID) developed by Reckase 
(I985). This definition defines MID as the direction and distance 
from the origin of the space to the point of steepest slope. 
Because of the close connection within IRT between item 
discrimination and item information, a multidimensional extension 
of the concept of item information will also be presented. 

This paper is composed of three parts. First, the concept of 
discrimination for multidimensional items will be developed in 
general terms. Second, the general definition will be applied to 
a particular multidimensional IRT model to determine the 
mathematical expression for multicimensional discrimination and 
information for that case. Finally, measures of multidimensional 
discrimination and information will be computed using item 
response data from a form of the ACT Mathematics U^^ge Test and 
the use of the statistics will be discussed. 



Theoretical Framework 
The work presented in this paper is based upon the assumption 
that the interaction between a person and an item can be described 
by one of a class of well behaved probability functions. These 
functions relate the probability of a correct response to an item 
to a person's location in a maltidimensional ability space, as 
indicated by their 9-vector, and the characteristics of the item, 
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as indicated by a vector of item parameters, a. That is, 

P (x^j = 1) = f (o., 9j) , (1) 

where Xj^j is the score on Item j. for Person J. This function is 
assumed to be "well behaved" in that for all dimensions in the 
space, or any combinations of dimensions, as Bj increases, 
P (Xj^j = 1) is nondecreasing. McKinley and Reckase (1982) 
describe a number of multidimensional item response theory (MIRT) 
models that are well behaved in the sense described here. 
Multidimensional Discrimination 

A measure of item discrimination for an item whose 
performance can be described by Equation 1 will be useful to the 
extent that it provides the same type of information that is 
provided by the unidimensional discrimination statistics. That 
is, the mult idimens ion measure of discrimination (MDISC) should 
allow items to be compared on a general measure of quality, to be 
classified as above or below a standard of quality, and to be used 
as an indicator of strength of relationship of the item 
performance to dimensions in the ability space. Further, it would 
be especially convenient if the MDISC statistics were related to 
the MID statistics in the same way that unidimensional IRT item 
statistics are related so that some of the interpretive framework 
that has been developed for the unidimensional case can be 
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generalized to the multidimensional case. 

In unidimensional IRT, the discrimination parameter is 
related to the slope of the item characteristic curve (ICC) at the 
point where the slope is steepest, the point of inflection. The 
point or inflection also defines the difficulty parameter for the 
items. The most direct generalization of thics concept to the 
multidimensional case would be to relate MDISC Lo the slope of the 
surface defined by Equation 1 at the point of steepest slope in 
the multidimensional space. However, this definition of MDISC has 
several problems. For some mathematical forms of the function in 
Equation 1 (e.g., the noncompensatory models), the point of 
maximum slope occurs when at least one of the e's approaches 
infinity. Therefore, the MDISC statistics could not be related to 
MID in the same way that is true for the unidimensional models. 
For other classes of models (e.g., the compensatory models), there 
are an infinite number of points of maximum slope. This fact also 
causes difficulties when interpreting the MDISC statistic. 
Therefore, an alternative definition for MDISC will be proposed 
that uniquely determines a single value for MDISC and that closely 
relates it to the MID. 

The definition proposed in this paper specifies the MDISC as 
a function of the slope of the item response surface (IRS) defined 
by Equation 1 at the steepest point in the direction indicated by 
the MID. Conceptually, determining the MDISC value can be thought 
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of as requiring several steps. First, a direction from the origin 
of the space is selected and the point of inflection of the IRS 
along the line defined by this direction is determined. This 
process determines the point of steepest slope in the direction 
specified. Next, the slope at the point of inflection in the 
specified direction is determined. The same process is followed 
in each direction from the origin and the steepest slope is 
determined. All of theses slopes are then compared to determine 
which is the steepest overall. This value will be used to compute 
the MDISC statistic. The direction that gives the steepest slope 
is the same as the direction specified by the MID. The distance 
from the origin to the point of steepest slope in the direction 
indicated by the MID, is the distance component of the MID 
statistic. 

The mathematical procedure for determining MDISC has four 
steps. First, the mathematical expression for the IRS is 
converted to polar coordinates to simplify the analysis. Second, 
the second derivative in direction a from the origin is used to 
determine the point of steepest slope in that direction. Third, 
the expression for the slope at the point of steepest slope is 
determined using the first derivative. Finally, the first 
derivative is taken with respect to a to determine the direction 
of overall steepest slope. A function of the rlope in that 
direction is proposed as the MDISC statistic. 
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For all of the IRS's considered so far, this procedure yields 
a unique value of MDISC that has the same relationship to MID that 
the a-parameter has to the b-parameter in unidimensional IRT. 
This relationship will be demonstrated in the next section when 
the MDISC definition is applied to a particular MIRT model. 
Multidimensional Information 

Although multidimensional information (MINF) is related to 
the MDISC in that if an item has a high value of MDISC it will 
provide a lot of information somewhere in the ability space, the 
concept is also quite different because it is concerned with the 
ability of the item to discriminate at each point in the space 
rather than Just at the steepest point of the IRS. 

The definition of MINF proposed in this paper is a direct 
generalization of the unidimensional IRT concept of information. 
For unidimensional IRT, information at an ability level, e, is 
defined as the ratio of the square of the slope of the ICC at 
ability level 9 to the variance of error of the item score at that 
level of e. Mathematically, item information is expressed as 
follows. 



1^(8) = 




(2) 



P^(8) Q.(9) 
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where 1^(9) is the information from Item i at ability level 8, 
Pj^(9) is the probability of a correct response to Item i 
for a person with ability 9, 
and 9^(9) 1 - P. (9) . 

Test information is simply the sum of item information values 

n 

U9) = I 1.(9) (3) 
i=l ^ 

where n is the number of items. 

For the multidimensional case, Equations 2 and 3 can still be 
used, but the slope in the numerator of Equation 2 must be 
determined in a slightly different way. When an IRS is considered 
instead of an ICC there are many slopes at any point in the 
ability space rather than one. Depending on the direction that is 
taken at the point in the space, the slope will differ. The slope 
will be much greater if a direction is selected that goes up the 
surface rather than one that goes across it. One direction may 
yield a slope of zero while another direction, at the same point 
in the space, may yield a fairly steep slope. Thus, direction in 
the space must be considered when determining the information 
provided by the item. This is the same as specifying how much 
information is provided about a particular composite of abilities 
at a point in the ability space. 
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In order to determine the slope in a particular direction, 
the mathematical procedure known as the directional derivative is 
needed. The directional derivative is defined as 

6 P(e) 6 P(e) 6 P(e) 

\ ^^^^ ' ~lBr ^ " a^* . . . + COS a ^, (U) 

^ * n 

where a is the vector of angles with the coordinate axes in 

the e-space, (i = l,n) is an element of the vector, 8 is the 

vector of abilities defining a point in the space, 

and B^{i = l,n) is an element of the vector. Equation U gives the 

slope in direction a at the point 6 in the ability space. 

Vfhen computing the multidimensional information, the 
directional derivative replaces the derivative in the numerator of 
Equation 2. When this is done, item information can be determined 
for any angle, representing different composites of abilities, in 
the space. Thus, to totally describe the information structure of 
an item, many information plots are needed, not just one. In 
principal, an information function can be determined for the 
infinite number of directions from the origin. In practice, 
determining the information function for angles at 10"* intervals 
between 0"* and 90® from the axes is sufficient to determine where, 
and for what combination of abilities, an item provides 
information. 

The next section of the paper presents a derivation of MDISC 
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and MINF for a particular MIRT model. The following section 
applies the concepts to actual test data from the ACT Mathematics 
Usage Test. 



MDISC and MIUF for the M2PL Model 
In order to demonstrate the use of MDISC and MINF, a MIRT 
model is needed that can be used to derive the mathematical 
expressions for the statistics. Since Reckase (1985) has already 
developed the MID concept using the multidimensional extension of 
the two-parameter logistic model (M2PL) and an estimation program 
is available for the model (McKinley 4 Reckase, 1983), the M2PL 
model will be used as an example. However, the concepts can also 
be applied equally as well to other MIRT models. 
The M2PL model is given by 



= 1 I a., d., Sj) 



1 + e 



aTSj . d. 



(5) 



where Xy is the score (0, 1) on Item i by Person J, 

a.^ is the vector of item discrimination parameters, 
dj^ is a scalar parameter that is related to the difficulty 
of the item 

and e is the vector of ability parameters for Person J. 
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MDISC 

The MDISC for an item is a function of the slope at the 
steepest point in the MID direction for an item. Reckase (1985) 
derived the MID direction as 



cos a 



Jik 



ik 



m 

^ ^ik 
Lk=1 



1/2 



(6) 



where a^^^^ is the angle with Axis k for Item i, 

a^j^ is lae kth element of vector a^^, 
and m is the number of dimensions in the space. 
He also determined that the slope of the IRS at the point of 
inflection in direction a^^ is 



1 



m 



(7) 



Substituting Equation 6 into Equation 7 yields the slope in the 
MID direction 



Slope 



1 "* 

li ^ -ik 
^ k=1 



?ik 



n 
Z 

Lk=1 



2ik 



1/2 



1/2 



(8) 



For the unidimensional two-parameter logistic (2PL) model, the 
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slope at the point of inflection is equal co (1.7) (1/4)aj^. 



Thus 



m 

z a?. 
.k=1 



1/2 



is analogous to the a-parameter in 



the unidimensional model. Therefore, it is proposed that the 
MDISC be defined as 



MDISC = 



m 1 1/2 
Z a?, 
k=1 



(9) 



This definition of MDISC has several nice properties. First, 
if an item measures only dimension I, that is, when a > 0 and 

= 0, 1 >^ 1, MDISC = a., and the multidimensional 
discrimination is equal to the unidimensional discrimination 
parameter, as it should. 

Second, when the 2PL model is expressed in the slope- 
intercept form, the exponent is given by a^9j > d^ 
where d, = -b.a.. The distance, D., in the MID has the same 
relationship with the intercept term, dj^, of Equation 5 as bj^ 
does with d, for the 2PL model. That is. 



d, = - D. • MDISC 
-i -1 



(10) 
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since 



- d, 



D. = 



m 



Lk=1 



-1 

1/2 ^ MDISC 



(11) 



Finally, MDISC Is on the same scale as aj^j^, namely, four 
times the slope, so it can be interpreted accordingly. The 
definition meets all the requirements s^'ated earlier for a 
generalization of the IRT disciumination parameter. 
MINF 

In order to compute the MINF for the M2PL model, the 
directional derivative of the IRS is needed. The directional 
derivative is given by 



V P.(e) 
a -1 



= a. ^P.(e)Q.(e)cos + ^^a^i^ ^^9i^ ^^^^^ ^2 ^ 

+ a. P.(e)Q.(e)cos a , 
-im -1 -1 m 



m 

= P.(e)Q.(e) I a cos a 



(12) 



This expression can be substituted for the term in the numerator 
of Equation 2 yielding 
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m 

(Pi(9) Q.(9) Z a.j^ cos aj^)2 

P.(9) Q.(9) 
m 

= P. (8) Q.(8) ( Z a., cos a. )2 

-1 -1 IK K 



(13) 



From this equation, the information at the point indicated by 9 in 
direction a can be determined. As with the unidimensional 
definition of information, the item information functions can be 
summed to obtain a test information function. However, when the 
test information is computed, the same direction must be used for 
all of the items. 

Example of the Application of MDISC and MINF 
In order to demonstrate the use of MDISC and MINF, Form 24B 
of the ACT Mathematics Usage Test was analyzed to determine 
estimates of the parameters of the M2PL model. The responses from 
a systematic sample of 1,000 examinees were used for this 
purpose. The MAXLOG program (McKinley i Reckase, 1983) was used 
to estimate the parameters. A two-dimensional solution was 
obtained for this example so that the results could be represented 
graphically . ' 

The parameter estimates for the M2PL model, the MID, and the 
MDISC statistics for the 40 items on the test are presented in Table 
1. Of the items on the test. Item 27 has the highest MDISC 
statistic. This means that of all of the items on the test, this 



Discriminating Power 
15 



item was tht best at differentiating between examinees in 
different parts in the e-space. However, this item discriminates 
best along a line that is at a U6' angle to the Dimension 1 
axis. Along the Dimension 1 axis (at 0» to the axis), the 
discrimination is only 1.66. which is less than the discrimination 
for IteiP 10 along the axis. Thga, the MDISC gives an overall 
measure of the quality of the item, but it do«5 not indicate that 
the item is of equal quality in measuring In all directions (I.e.. 
for all weighted composites of abilities). 



Insert Table 1 about here 



The MDISC statistics for two items can only be directly 
compared if the itens measure in the same direction. For example, 
Items 29 and 36 can be compared on MDISC because they measure in 
the same direction. The MDISC statistics cannot be compared for 
Items 3 and 30 because the directions are quite different. To 
compare these items, a common dire ' 'on o, would have to be 
selected, and then the discrimination in the a direction would 
have to be computed using the formula 



Directional Discrimination = C a^^^ cos o^^^ (1^0 

ksl 



For Items 3 and 30, the directional discriminations in direction 
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30'' from Dimension 1 are 1.54 and 1.19 respectively: at 60** from 
Dimension 1, they are 1.10 and 1.39 respectively. Thus, depending 
on the direction, the ordering of the items on discrimination 
changes. However, Item 3 is more discriminating overall since it 
has a higher MDISC statistic. 

In order to give some further guidance in interpreting the 
MDISC statistic, the correlation has been computed between it and 
the biserial correlation between the item and the total score on 
the test, the a-parameter estimate from the three-parameter 
logistic model obtained from LOGIST (Wingersky, Barton i Lord, 
1982), and the a-parameter estimates from the M2PL model. These 
correlations are given in Table 2. The MDISC statistic for this 
set of data was found to be correlated most highly with the r^^^ 
statistic. It is interesting that the a-parameter estimate from 
LOGIST is most highly related to aj while r^^^ is most highly 
related to a^ . The relationship between MDISC, and a^ and §2 is 
dictated by Ec^uation 9. 



Insert Table 2 about here 



In order to demonstrate the use of the MINF, the MINF was 
computed for Item 10 using directions of 0% 30% 60% and 90'* 
from Dimension 1. The results for this item are shown in two 
different ways in Figures 1 and 2. Figure 1 indicates the amount 
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of information in direction a by the height of the surface above 
the 9-plane. The four parts of Figure 1 show the surfaces for 
each of the four directions. The figures show that the item gives 
no information about Dimension 2, and that the amount of 
information provided by the item increases as the angle goes from 
90^ to 0°. In all cases, the information is greatest along the 
line, 8- = -.19. 



Insert Figure 1 about here 



This same data is presented in Figure 2 using a 
representation scheme suggested by David Thissen. At selected 
points in the 8-space, the information is represented by the 
length of the line in the direction taken in the space. Lines are 
give at 10^ intervals. Figure 2 shows that Item 10 gives no 
information about Dimension 2, and progressively more information 
as the angle goes from 90° to 0*^ with respect to Dimension 1. 
Most of the information provided in the four parts of Figure 1 is 
given in Figure 2. 



Insert Figure 2 about here 



The information supplied by the entire test is shown by the 
three surfaces in Figure 3 and the line plots in Figure 4. A 
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comparison of Figures 3a and 3c shows that the test supplies 
somewhat more information about than Figure 3b shows that 

the most information is provided about an equally weighted 
composite of 8^ and B^, The same information is given in Figure 4, 
but the line plot more clearly indicates the regions of 
the 9-space that are best measured by the test. 



Insert Figures 3 and 4 about here 



Discussion 

The purpose of this paper has been to define conceptually the 
concepts of multidimensional discrimination and information, to 
derive the mathematical expressions for the concepts for a 
particular multidimensional IRT model, and then to apply the 
concepts to actual test data. Multidimensional discrimination was 
defined as a function oi the slope of the item response surface in 
the direction specified by the multidimensional difficulty 
(Reckase, 1985). For the M2PL model, this definition results in a 
statistic that has the same relationship to multidimensional 
difficulty that a does to b for the unidimensional IRT models. 
This statistic is defined as the square root of the sum of the 
squared a-parameters from the M2PL model. 
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Multidimensional information was defined by the same 
mathematical function as unidimensional information, but the 
directional derivative was substituted for the standard derivative 
in the numerator of the information expression (Equation 2). The 
use of the directional derivative results in an information 
measure that corresponds to a direction in the ability space. 
Direction, in this case, is an indicator of the composite of 
abilities that is of interest. Use of multidimensional 
information makes very clear the dimensions being measured by a 
test and the regions of the ability space being measured on each 
dimension. 

The ACT Mathematics Usage Test was used to demonstrate the 
multidimensional discrimination and information. The use of the 
statistics show that the items on the test vary substantially in 
their directions of maximum discrimination and that the test tends 
to measure Dimension 1 or a combination of Dimensions 1 and 2 at a 
greater level of precision than Dimension 2 alone. The regions of 
the ability space that are best measured were also clearly 
indicated. 

The multidimensional measures of item quality, item 
precision, and test precision given in this paper provide another 
set of tools that can be used to gain a better understanding of 
the measureu.ent process. Through their use, and the use of 
multidimensional models, the dominance of unidimensional measures 
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may finally by broken, leading to a more powerful class of 
measurement devices. 
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Table 1 

Item parameters, MID, and MDISC for the items in the ACT 
Mathematics usage Test, Form 24B 



MID 



tein 


a- « 


a • ^ 


d- 
1 


il 


i2 


D- 
1 


MDISC 


1 


1.81 


.86 


1.46 


25 


65 


-.73 


2.00 


2 


1.22 


.07 


.17 


4 


89 


-.14 


1.22 


3 


1.57 


.36 


.67 


13 


77 


-.42 


1.61 


4 


.71 


.53 


.44 


37 


53 


-.50 


.89 


5 


.86 


.19 


.10 


12 


78 


-.11 


.88 


6 


1.72 


.18 


.44 


6 


84 


-.25 


1.73 


7 


1.86 


■ .29 


.38 


9 


81 


-.20 


1.88 


8 


1.33 


.34 


.69 


14 


76 


-.50 


1.37 


9 


1.19 


1.57 


.17 


53 


37 


-.09 


1.97 


10 


2.00 


.00 


.38 


0 


90 


-.19 


2.00 


11 


.87 


.00 


.03 


0 


90 


-.03 


.87 



(table continues) 
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MID 



Item a^^ a^g "il ° i2 °i 



12 


2.00 


.98 


.91 


26 


64 


-.41 


2.23 


13 


1.00 


.89 


-.49 


42 


48 


.37 


1 .34 


14 


1.22 


.14 


.54 


7 


S3 


-.44 


1.23 


15 


1.27 


.47 


.29 


20 


70 


-.21 


1.35 


16 


1.35 


1.15 


-.21 


40 


50 


.12 


1.77 


1 7 
1 f 






nfl 
. uo 




fi7 

0 1 






18 


1.92 


.00 


.12 


0 


90 


-.06 


1.92 


19 


.96 


.22 


-.30 


13 


77 


.30 


.98 


20 


1.20 


.12 


-.28 


6 


84 


.23 


1 .21 


21 


1.41 


.04 


-.21 


2 


88 


.15 


1.41 


22 


1.54 


1.79 


.02 


49 


41 


-.01 


2.36 


23 


.54 


.23 


-.69 


23 


67 


1.18 


.59 


24 


1.53 


.48 


-.83 


17 


73 


.52 


1 .60 


25 


.72 


.55 


-.56 


37 


53 


.62 


.91 


26 


.51 


.65 


-.49 


52 


38 


.59 


.83 


27 


1.66 


1.72 


-.38 


46 


44 


.16 


2.39 



(table continues) 
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Discriminating Power 

2U 



MID 



Item a^^ a^g ^12 MDISC 



28 


.69 


. "9 


-.68 


15 


75 


95 


.72 


29 


.88 


1.12 


-.91 


52 


38 


.64 


1.42 


30 


.68 


1 .21 


-1 .08 


61 


29 


.78 


1 .39 


31 


.2!* 


1.14 


-.95 


78 


12 


.82 


1.36 


32 


,51 


1.21 


-1.00 


67 


23 


.76 


1.31 


33 


.76 


.59 


-.96 


38 


52 


1 .00 


.96 


3M 


.01 


1.94 


-1.92 


90 


0 


.99 


1.94 


35 


.39 


1.77 


-1.57 


78 


12 


.87 


1.81 


36 


.76 


.99 


-1.36 


52 


38 


1.09 


1.25 


37 


.19 


1.10 


-.81 


66 


24 


.67 


1.20 


38 


.29 


1.10 


-.99 


75 


15 


.87 


1.14 


39 


.48 


1 .00 


-1.56 


64 


26 


1.41 


1.11 


UO 


.U2 


.75 


-1.61 


61 


29 


1.87 


.86 
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Table 2 

Correlations betwee.. discrimination parameter estimates 



Dlscrlfliination 

Estimate 1. 2. 3. M. 5. 



1. MDISC .46 .78 .62 .46 

^' ^LOGIST '^^ •^'^ 

3. rgis .80 .16 

U, a^ --.34 

5. aj 
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Figure Caption 

Figure U Multidimensional information for Item 10 at angles of 
0*, 30*, 60^, and 90"^ to the 9j axis. 

Figure 2 . Multidimensional information for Item 10 represented by 

directional vectors from points in the 9-space. 

Figure 3 * Multidimensional information for the ACT Mathematics 

Usage Test at angles of 0% 45"*, and 90"" to the 6^ axis. 

Figure Multidimensional information for the ACT Mathematics 

Usage Test represented as directional vectors from points iti the 

9-space. 
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FIGURE 2 
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